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EXECUTIVE SUMMARY 

The following report gives the statistical findings of the 2009-2010 Watson-Glaser test. 
Data is pre-existing and was given to the Evaluator by email from the Director, Center 
for Legal Pedagogy. Statistical analyses were run using SPSS 17 to address the 
following questions: 

1 . What are the statistical descriptors of the Watson-Glaser results of student Pre- 
Test and Post-Test (Within and Between)? 

2. What is the relationship of the categories (Inference Making - IM, Assumption 
Recognition - AR, Deductive Reasoning - DR, Interpretation - IN, and Argument 
Evaluation - AE) of the Watson-Glaser Pre and Post Test data? 

3. What categories indicated statistically significant differences from Pre to Post 
Test? 

Findings, summaries, and conclusive statements are also written as side notes within 
the appendices of this report/study. Those and the following summaries specifically 
address each research question and possible implications. 

Procedures 

A group of students independently took the Watson-Glaser Pre-Test (N=200) in August 
2009 during the orientation week for their 1 st year as TMSL students. Of that same 
group, 90 independently took the Post Test towards the end of the semester. The data 
was matched to the individuals and the findings are reported: 

Findings 

Summary of Findings for Research Question 1 : The Median measure is the best 
indicator of group data in this situation. Hence the Medians Total Score reported are 59 
for Pre-Test Takers who took Post-Test (N=90), 59 for Pre-Test Takers whom did not 
take the Post -Test (N=1 10), and a Post-Test Median Final Score of 50. It was found 
that the Final Score Median declined 9 points from Pre to Post Tests results. (See the 
Histograms in Appendix 1 for a better visual description of Data.) 

Summary of Findings for Research Question 2: The relationship measure here is 
indicated by a paired sample correlations (N=90). This measure could only be given 
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from those whom took both the Pre and Post Test. There were 2 significant 
relationships: The AR-AR2 and the AE-AE2 (p<.05). Please note that the other 
correlational coefficients were too small to indicate any significant level of relationship. 
Also note there is a negative yet insignificant relationship with DR-DR2. 

Summary of Research Question 3: The statistically significant measure here is indicated 
by a paired sample tests (N=90). This measure could only be given from those whom 
took both the Pre and Post Test. There were statistically significant differences given 
for ALL of the measured Pre and Post Tests categories when using a Paired Sampled 
Test (see Appendix 3). It is necessary to note that these differences are non directional 
but an investigation of the means would indicate that the significant difference was 
negative. After further investigation by the evaluator it was determined that the Pre and 
Post Tests were not administered with the same standards. This difference can also 
show up on the data when a very valid and highly reliable instrument is used (such as 
the Watson-Glaser). It is a final recommendation that future administrations of the tests 
follow similar protocols in order to provide more useful data. 

Conclusions 

The findings this report will be used as information in Professional Development 
workshops in order to help Faculty/Staff see the impact of the Law School 1 st year 
process on the student’s inference Making - IM, Assumption Recognition - AR, 
Deductive Reasoning - DR, Interpretation - IN, and Argument Evaluation - AE skills. 

All of which are considered high need skills for a successful lawyer. 
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Appendix 1 



Findings for Question 1 : What are the statistical descriptors of the Watson-Glaser 
results of student Pre-Test and Post-Test (Within and Between)? 



Test Takers Statistics for Watson Glaser August 2009 





IM 


AR 


DR 


IN 


N 


Valid 


90 


90 


90 


90 




Missing 


0 


0 


0 


0 


Mean 




9.87 


12.32 


11.56 


12.62 




Median 




10.00 


13.00 


12.00 


13.00 


Std. Deviation 


2.423 


3.097 


2.249 


1.858 


Skewness 


.004 


-1.704 


-.456 


-.633 


Std. Error of Skewness 


.254 


.254 


.254 


.254 


Minimum 


5 


0 


6 


8 


Maximum 


15 


16 


15 


16 


Percentiles 25 


8.00 


12.00 


10.00 


11.00 




50 


10.00 


13.00 


12.00 


13.00 




75 


12.00 


14.00 


13.00 


14.00 





AE 


TOT 


N 


Valid 


90 


90 




Missing 


0 


0 


Mean 




12.26 






Median 




13.00 


59.00 


Std. Deviation 


3.161 


7.018 


Skewness 


-2.321 


-.369 


Std. Error of Skewness 


.254 


.254 


Minimum 


0 


41 


Maximum 


16 


74 


Percentiles 25 


12.00 


54.00 




50 


13.00 


59.00 




75 


14.00 


64.00 
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Non-Test Takers of Post Test Statistics for Watson 
Glaser August 2009 





IM 


AR 


DR 


IN 


N 


Valid 


110 


110 


110 


110 




Missing 


0 


0 


0 


0 


Mean 




9.99 


11.80 


11.41 


12.11 




Median 




10.00 


13.00 


12.00 


12.00 


Std. Deviation 


2.414 


3.850 


2.700 


2.207 


Skewness 


-.069 


-1.225 


-1.106 


-.725 


Std. Error of Skewness 


.230 


.230 


.230 


.230 


Minimum 


4 


1 


0 


4 


Maximum 


15 


16 


16 


16 


Percentiles 25 


8.00 


10.00 


10.00 


11.00 




50 


10.00 


13.00 


12.00 


12.00 




75 


11.00 


15.00 


13.00 


14.00 
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Watson Glaser Post Test Ta 


kers Spring 201 


0 




IM2 


AR2 


DR2 


IN2 


N Valid 


90 


90 


90 


90 


Missing 


0 


0 


0 


0 


Mean 


7.34 


10.86 


9.63 


10.56 


Median 


p.oq 


11.00 


10.00 


11.00 


Std. Deviation 


2.748 


3.157 


2.628 


3.145 


Skewness 


-.177 


-.743 


-.676 


-.525 


Std. Error of Skewness 


.254 


.254 


.254 


.254 


Minimum 


1 


2 


0 


3 


Maximum 


13 


16 


16 


16 


Percentiles 25 


5.00 


9.00 


8.00 


8.75 


50 


8.00 


11.00 


10.00 


11.00 


75 


10.00 


13.00 


11.00 


13.00 





AE2 


TOT2 


N 


Valid 


90 


90 




Missing 


0 


0 


Mean 




10.41 


48.80 




Median 




11.00 


50.00 


Std. Deviation 


2.548 


9.853 


Skewness 


-.236 


-.745 


Std. Error of Skewness 


.254 


.254 


Minimum 


4 


21 


Maximum 


16 


70 


Percentiles 25 


9.00 


43.75 




50 


11.00 


50.00 




75 


12.00 


56.25 



★★★★★★★★★★ 

Summary of Research 
Question 1 : The Median 
measure is the best 
indicator of group data in 
this situation. Hence the 
Medians Total Score 
reported are 59 for Pre- 
Test Takers who took 
Post-Test (N=90), 59 for 
Pre-Test Takers whom 
did not take the Post - 
Test (N=1 1 0), and a Post- 
Test Median Final Score 
of 50. Note, that the Final 
Score Median declined 9 
points from Pre to Post 
Tests. 

See the following 
Histograms for a better 
visual description of Data. 

★★★★★★★★★★ 
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Frequency Frequency Frequency 



20- 



IM 



Mean = 9.87 
Std. Dev. = 2.423 
N = 90 




IM 



IM2 




Mean = 9.99 
St cl . Dev. = 2.41 4 
N = 1 1 O 




Mean = 7.34 
Std. Dev. = 2.748 
N = 90 



1 st Histogram 
represents Pre-Test 
Takers whom took the 
Post-Test (N=90) 



2 nd Histogram 
represents Pre-Test 
Takers whom did not 
take the Post Test 
(N=110) 



3 rd Histogram 
represents Post-Test 
Takers (N=90) 



2.5 



7.5 

I M2 



10 
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Frequency Frequency Frequency 



AR 



Mean = 12.32 
Std. Dev. = 3.097 
N = 90 




1 st Histogram 
represents Pre-Test 
Takers whom took the 
Post-Test (N =90) 



AR 




2 nd Histogram 
represents Pre-Test 
Takers whom did not 
take the Post Test 
(N=110) 



AR2 




3 rd Histogram 
represents Post-Test 
Takers (N=90) 



10 

AR2 



is 
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Frequency Frequency Frequency 



DR 



Mean = 1 1 .56 
Std. Dev. = 2.249 
N = 90 




DR 





o 



5 



15 



Mean = 9.63 
Std. Dev. = 2.628 
N = 90 



1 st Histogram 
represents Pre-Test 
Takers whom took the 
Post-Test ( N=90) 



2 nd Histogram 
represents Pre-Test 
Takers whom did not 
take the Post Test 
(N=110) 



3 rd Histogram 
represents Post-Test 
Takers (N=90) 



DR2 



10 
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Frequency Frequency Frequency 



IN 



Mean = 12.62 
Std . Dev. = 1 .858 
N = 90 




1 st Histogram 
represents Pre-Test 
Takers whom took the 
Post-Test (N=90) 



IN 




Mean = 12.11 
Std. Dev. = 2.207 
N = 1 1 0 

2 nd Histogram 
represents Pre-Test 
Takers whom did not 
take the Post Test 
(N=110) 



IN2 




3 rd Histogram 
represents Post-Test 
Takers (N=90) 



10 

IN2 



15 
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Frequency Frequency Frequency 



AE 




AE 




AE2 




1 st Histogram 
represents Pre-Test 
Takers whom took the 
Post-Test (N=90) 



2 nd Histogram 
represents Pre-Test 
Takers whom did not 
take the Post Test 
(N=110) 



3 rd Histogram 
represents Post-Test 
Takers (N=90) 



5 



AE2 
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Frequency Frequency Frequency 



TOT 




TOT 





Mean = 48.8 
Std. Dev. = 9.853 
N = 90 



1 st Histogram 
represents Pre-Test 
Takers whom took the 
Post-Test (N=90) 



2 nd Histogram 
represents Pre-Test 
Takers whom did not 
take the Post Test 
(. N=110 ) 



3 rd Histogram 
represents Post-Test 
Takers (N=90) 



20 



TOT2 



60 
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Appendix 2 



Findings for Research Question 2: What is the relationship of the categories 
(Inference Making - IM, Assumption Recognition - AR, Deductive Reasoning - 
DR, Interpretation - IN, and Argument Evaluation - AE) of the Watson-Glaser Pre 
and Post Test data? 



Paired Samples Statistics for Watson-Glaser 




Paired Samples Correlations 



Pair 1 
Pair 2 
Pair 3 
Pair 4 
Pair 5 
Pair 6 



IM & IM2 
AR & AR2 
DR & DR2 
IN & IN2 
AE & AE2 
TOT & TOT2 



90 

90 

90 

90 

90 

90 



Correlation 



.054 

.222 

-.102 

.031 

.207 

.090 



Sig. 




£359 

.339 

2 

.397 



★★★★★★★★★★ 

Summary of Research 
Question 2: The 
relationship measure here 
is indicated by a paired 
sample correlations 
(N=90). This measure 
could only be given from 
those whom took both the 
Pre and Post Test. There 
were 2 significant 
relationships: The AR- 
AR2 and the AE-AE2 
(p<05). Please note that 
the other correlational 
coefficients were too 
small to indicate any 
significant level of 
relationship. Also note 
there is a negative yet 
insignificant relationship 
with DR-DR2. 

★★★★★★★★★★ 
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Appendix 3 



Findings for Research Question 3: What categories indicated statistically significant 
differences from Pre to Post Test? 



Paired Samples Test 





Paired Differences 


Mean 


Std. Deviation 


Std. Error Mean 


95% Confidence Interval of the 
Difference 


Lower 


Upper 


Pair 1 


IM- IM2 


2.522 


3.564 


.376 


1.776 


3.269 


Pair 2 


AR - AR2 


1.467 


3.901 


.411 


.650 


2.284 


Pair 3 


DR- DR2 


1.922 


3.630 


.383 


1.162 


2.682 


Pair 4 


IN - IN2 


2.067 


3.603 


.380 


1.312 


2.821 


Pair 5 


AE - AE2 


1.844 


3.625 


.382 


1.085 


2.604 


Pair 6 


TOT - TOT2 


9.811 


11.569 


1.219 


7.388 


12.234 





T 


df 


Sig. (2-tailed) 


Pair 1 


IM - IM2 


6.714 


89 


.000 


Pair 2 


AR - AR2 


3.567 


89 


.001 


Pair 3 


DR- DR2 


5.024 


89 


.000 


Pair 4 


IN - IN2 


5.441 


89 


.000 


Pair 5 


AE - AE2 


4.826 


89 


.000 


Pair 6 


TOT - TOT2 


8.045 


89 


.000 



★★★★★★★★★★ 

Summary of Research Question 3: The statistically significant measure here is indicated 
by a paired sample tests (N=90). This measure could only be given from those whom 
took both the Pre and Post Test. There were statistically significant differences given for 
ALL of the measured Pre and Post Tests categories. It is necessary to note that these 
differences are non directional but an investigation of the means would indicate that the 
significant difference was negative. After further investigation by the evaluator it was 
determined that the Pre and Post Tests were not administered with the same standards. 
This difference can also show up on the data when a very valid and highly reliable 
instrument is used (such as the Watson-Glaser). It is a final recommendation that future 
administrations of the tests follow similar protocols in order to provide more useful data. 

★★★★★★★★★★ 
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